Wherefore, what is claimed is: 

1 . A computer-implemented process for outputting whiteboard 
content, comprising the following process actions: 

inputting a sequence of image frames of content written on a whiteboard 
in real-time; 

dividing each of said image frames into cells; 
for each cell in each image frame, 

determining if there is a change in the cell image compared to a 
correspondingly located cell the immediately preceding image frame in said 
sequence of image frames; 

whenever it is determined that there is a color change, setting a cell 
age to a prescribed minimum value, and if there is no change increasing cell age 
by a prescribed increment value; 

determining if cell age is greater than a specified threshold value; 
whenever the cell age is not greater than the threshold value, not 
processing said cell any further; 

if said cell age is greater than the threshold value, 
computing the background color of the cell; 
updating a whiteboard color model; 
classifying each cell image as a foreground or whiteboard cell using 
said whiteboard color model; 

whenever a cell image is classified as a foreground cell not 
processing that cell any further; and 
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whenever a cell image is classified as a whiteboard cell, outputting 
the cell image whenever it exhibits strokes not present in the correspondingly 
located cell in the preceding image frames or is missing strokes found in the 
preceding image frames in real-time. 

2. The computer-implemented process of Claim 1 wherein the image 
frames are input in Bayer format. 

3. The computer-implemented process of Claim 1 further comprising 
the process action of outputting the image frame cells so that the strokes on the 
whiteboard are displayed in an enhanced manner using said cell classification. 

4. The computer-implemented process of Claim 3 wherein the 
process action of outputting the frame cells so that the strokes on the whiteboard 
are displayed in an enhanced manner comprises the process actions of: 

making the whiteboard background color for cells displaying whiteboard 
background in said sequence of image frames more uniform; 

increasing the stroke saturation of the whiteboard content to make the 
strokes more vivid and legible; and 

reducing image noise in said sequence of image frames. 
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5. The computer-implemented process of Claim 4 wherein the 
process action of outputting a cell comprises outputting only changed whiteboard 
cells. 

6. The computer-implemented process of Claim 1 , wherein said cell 
classifying process action comprises: 

comparing the input image frame cell to a previous image in a 
corresponding cell location by comparing the difference between the background 
color of the cell image and the corresponding color of the whiteboard model; 

if the difference exceeds a given threshold, designating the cell as a 
foreground cell, and if the difference does not exceed a given threshold 
designating the cell as a whiteboard cell; 

determining if the cell location was a whiteboard cell in a previous output, 
and if not designating the cell as a new whiteboard cell and outputting its cell 
image; 

if the cell location was not a whiteboard cell in the previous output, 
determining if the cell image in the same location of the output is the same and if 
not outputting its cell image. 

7. The computer-implemented process of Claim 6 wherein the 
difference between a background color of the cell image and the corresponding 
color in the whiteboard color model is determined by determining the Euclidean 
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distance between the cell image and the corresponding cell location in the 
whiteboard color model. 

8. The computer-implemented process of Claim 6 further comprising 
the process action of verifying that said cell classified as a whiteboard cell is a 
whiteboard cell, comprising the process actions of: 

determining whether said whiteboard cell is connected to any foreground 
cells; and 

if said whiteboard cell is determined to be connected to any foreground 
cells, classifying it as a foreground cell. 

9. The computer-implemented process of Claim 6 further comprising 
the process action of verifying that said cell classified as a foreground cell is a 
foreground cell, comprising the process actions of: 

determining whether said foreground cell is connected to other foreground 
cells; and 

if a cell is determined to be a foreground cell and not connected to other 
cells, classifying it as a whiteboard cell. 

1 0. The computer-implemented process of Claim 1 wherein the 
process action of updating the whiteboard color model for the whiteboard and 
stroke cells comprises the process actions of: 

dividing the image of the whiteboard into cells; 
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sorting the pixels in each cell according to their luminance value; and 
assigning the highest luminance value in each cell as the resulting 
whiteboard color of that cell. 

11. The computer-implemented process of Claim 1 0 further comprising 
the process action of filtering the colors of the cells comprising the following 
process actions: 

subjecting the resulting cell colors to a least-median-square error 
procedure, which fits a global plane over the colors and throws away the cells 
that are foreground cells; 

designating the remaining cells as whiteboard background cells; 

using said remaining whiteboard background cells to update the 
whiteboard background; 

and filling the holes in the whiteboard color model created by throwing out 
the cells that were foreground cells with known colors from neighboring cells with 
whiteboard background colors. 

12. The computer-implemented process of Claim 1 wherein the 
process action of updating the whiteboard color model comprises the process 
actions of: 

dividing the image of the whiteboard into cells; 

sorting the pixels in each cell according to their luminance value; and 
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averaging the top 10% values of the luminance values in each cell and 
assigning this value as the resulting whiteboard color of that cell. 

13. The computer-implemented process of Claim 1 further comprising 
correlating said audio recordings with said video. 

14. The computer-implemented process of Claim 1 wherein 
determining if there is a change in said cell image is computed in Bayer color 
space. 

1 5. The computer-implemented process of Claim 1 wherein the 
background color of the cell is computed in Bayer color space. 

16. The computer-implemented process of Claim 1 wherein the 
whiteboard color model is computed in Bayer color space. 

17. The computer-implemented process of Claim 1 wherein classifying 
cell images is computed in Bayer color space. 

18. The computer-implemented process of Claim 1 wherein 
determining if cell images are the same comprises using a normalized cross- 
correlation technique to compare two cells at a time. 
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19. The computer-implemented process of Claim 1 8 wherein the cross- 
correlation score ranges from -1 , for two images not similar at all, to 1 , for two 
identical images. 

20. The computer-implemented process of Claim 1 9 further comprising 
a process action of applying a Mahalanobis distance test to determine if two cells 
are the same. 

21 . The computer-implemented process of Claim 20 wherein the 
Mahalanobis distance test given by d = \i (o + jy, where / is the first cell 

image and/' is the second cell image, 7 is the mean color of the first cell image, 
T is the mean color of the second cell image, a is the standard deviation from 7 
and <r' is the standard deviation from r and wherein I and I' are the two cell 
images are considered to be identical if and only if d < T d ando T c , and wherein 

T d = 2 and t c = 0.707 . 

22. The computer-implemented process of Claim 1 wherein the 
whiteboard color model is dynamically updated by incorporating the background 
color from the current frame into the existing whiteboard color model. 

23. The computer-implemented process of Claim 22 wherein the 
background color from the whiteboard color model is updated using a Kalman 
filter technique. 
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24. The computer-implemented process of Claim 1 further comprising 
the process action of analyzing the sequence of images to isolate key frames 
summarizing key points of said whiteboard content. 

25. A system for transferring the content of data written on a 
whiteboard to at least one remote meeting participant, comprising: 

a capture system that captures a sequence of images of content written 
on a whiteboard and audio signals corresponding to sounds that occur during a 
meeting; 

an analysis server for analyzing the sequence of images and transmitting 
whiteboard content without foreground obstructions and associated audio to a 
remote meeting participant in real-time wherein the analysis server comprises 
modules for: 

dividing each of said images into cells of cell images; 
for each image, 

determining if the cell images have changed from the last frame; 

computing the background color of the cell images which represent 
the color of blank whiteboard; 

updating a whiteboard background model by integrating the color of 
the blank whiteboard computed from the cell images; 

classifying the cell images into foreground or whiteboard cells using 
the whiteboard color model; 

extracting changed whiteboard content; and 
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outputting changed whiteboard content to at least one remote 
participant. 

26. The system of Claim 27 further comprising a module for enhancing 
5 the color of whiteboard content. 

27. The system of Claim 25 wherein each image frame is processed in 
Bayer color format. 

10 28. The system of Claim 27 wherein the sequence of images are 

captured in Bayer color format. 

29. The system of Claim 25 wherein the analysis server processes and 
outputs changed whiteboard content in real-time. 

15 

30. A computer-readable medium having computer-executable 
instructions for transferring whiteboard content captured during a meeting in real- 
time, said computer executable instructions comprising program modules for: 

capturing a sequence of images of content written on a non-electronic 
20 whiteboard in Bayer color format; 

recording audio signals correlated with the sequence of images; and 
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analyzing the sequence of images to transmit said images of the 
whiteboard and audio signals to at least one remote meeting participant by: 

dividing each of said image frames into cells of cell images; 

determining if there is a change in the cell image compared to a 
correspondingly located cell the immediately preceding image frame in said 
sequence of image frames; 

whenever it is determined that there is a color change, setting a cell 
age to a prescribed minimum value, and if there is no change increasing cell age 
by a prescribed increment value; 

determining if cell age is greater than a specified threshold value; 

whenever the cell age is not greater than the threshold value, not 
processing said cell any further; 

if said cell age is greater than the threshold value, 
computing the background color of the cell; 
updating a whiteboard color model; 

classifying each cell image as a foreground or whiteboard cell using 
said whiteboard color model; 

whenever a cell image is classified as a foreground cell not 
processing that cell any further; and 

whenever a cell image is classified as a whiteboard cell, outputting 
the cell image whenever it exhibits strokes not present in the correspondingly 
located cell in the preceding image frames or is missing strokes found in the 
preceding image frames in real-time. 
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31 . The computer-readable medium of Claim 30 further comprising a 
module for correcting for exposure if more than a prescribed percentage of the 
cells have changed. 

32. The computer-readable medium of Claim 31 wherein the 
prescribed percentage of cells is 90 percent. 

33. The computer-readable medium of Claim 31 wherein the module 
for correcting for exposure automatically sets the appropriate amount of light to 
be captured to form each image frame in order to avoid over or underexposing 
resulting image frame. 

34. The computer-readable medium of Claim 31 wherein the module 
for correcting for exposure comprises sub-modules for: 

initializing the lower and upper limits of a search range of a search 
algorithm to be the entire range of the exposure setting; 

setting the search range to the average of the lower and upper limits; 
measuring over-exposure and under-exposure for each incoming image 

frame; 

exiting the module for correcting exposure if a given incoming image 
frame is neither over-exposed or under-exposed; 
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if an incoming image frame is over exposed, setting the upper searching 
limit to the current exposure setting; 

if an incoming image frame is under exposed, setting the lower searching 
limit to the current exposure setting; and 

if an incoming image frame is either over exposed or under exposed, 
setting the exposure of the incoming image frame to the average of the updated 
lower and upper limits. 

35. A computer-implemented process for creating an enhanced data 
stream content written on a piece of paper, comprising the following process 
actions: 

inputting a sequence of image frames of content written on the document 
in real-time; 

dividing each of said image frames into cells of cell images; 

for each cell location in each image frame, 

determining if there is a change in said cell image from the cell in 
the same location in the previous image frame in said sequence of image 
frames; 

if there is a change, setting cell age to a minimum value, and if 
there is no changes increasing cell age by a prescribed value; 

determining if cell age is greater than a specified value; 

if cell age is not greater than a specified value, not processing said 
cell further for said image frame; 
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if said cell age is greater than a specified value, computing the 
background color of the cell; 

updating a document color model; 

classifying cell images into foreground or document cells using said 
document color model; 

if a cell image is classified as a foreground cell not processing that 
cell any further for that input image frame; and 

if a cell image is not classified as a foreground cell, outputting any 
cell image frames with newly appeared strokes or newly deleted strokes in real- 
time. 
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